NONPARAMETRIC FUNCTIONAL MAPPING OF QUANTITATIVE TRAIT LOCI By JIE YANG A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NONPARAMETRIC FUNCTIONAL MAPPING OF QUANTITATIVE TRAIT LOCI By Jie Yang August 2006 Chair: George Casella Major Department: Statistics Functional mapping is a tool for detecting major genes responsible for different phenotypic curves, developed by Ma, Casella and Wu. The methodology uses a parametric functional form, usually derived from a biological law, to drive a maximum-likelihood-based test for a significant QTL (quantitative trait loci). However, in many situations there is no obvious functional form and, in such cases, this strategy will not be optimal. In this dissertation we propose to use nonparametric function estimation, typically implemented with B-splines, to estimate the underlying functional form of phenotypic trajectories, and then construct a nonparametric test to find evidence of existing quantitative trait loci. Using the representation of a nonparametric regression as a mixed model, we can easily derive a likelihood ratio test statistic. Two situations are considered: one is based on the dense-map assumption that QTL (quantitative trait loci) is on some marker and the other is the situation where the actual genes responsible for different underlying phenotypic trajectories might not just be on a marker locus, which is more realistic. For the dense-map case, after obtaining the joint distribution of all test statistics at every putative